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Abstract 

Estimating the condition numbers of random structured matrices is a well known chal- 
lenge (cf. |SST06p . linked to the design of efficient randomized matrix algorithms in |PGMQ| , 
[P1MR10| . |PQ10| , |PQ12| , |PQZa| , [P^a] , |PQZb| , |PQZC| , |PY09| . We deduce such estimates 
for Gaussian random Toeplitz and circulant matrices. The former estimates can be surprising 
because the condition numbers grow exponentially in n as n — > oo for some large and important 
classes of n x n Toeplitz matrices [ B G05] , whereas we prove the opposit for Gaussian random 
Toeplitz matrices. Our formal estimates are in good accordance with our numerical tests, except 
that circulant matrices tend to be even better conditioned according to the tests than according 
to our formal study. 

2000 Math. Subject Classification: 15A52, 15A12, 65F22, 65F35 
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1 Introduction 

It is well known that random matrices tend to be well conditioned |D88j . |E88j . |ES05j . [CD05 , 
SST06], [Bllj . and this property can be exploited for advancing matrix computations (see e.g., 
IPGMQI , [PIMRlOj . [PQTOl , |PGjl2l , [PQal , |PQZa| , |PQZb| , |PQZC| (PY09] 1. Exploiting matrix 
structure in these applications was supported empirically in the latter papers and formally in [Tllj . 
An important step in this direction is the estimation of the condition numbers of structured matrices 
stated as a challenge in SST06 . We reply to this challenge by estimating the condition numbers of 
Gaussian random Toeplitz and circulant matrices both formally (see Sections [5] and [7]) and experi- 
mentally (see Tables rSTTI I8.4[) . Our study shows that Gaussian random Toeplitz circulant matrices 
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do not tend to be ill conditioned and the condition numbers of Gaussian random circulant n x n 
matrices tend to grow extremely slow as n grows large. Our numerical tests (the contribution of 
the second author) are in good accordance with our formal estimates, except that circulant matrices 
tended to be even better conditioned in the tests than according to our formal study. Our results 
on Toeplitz matrices are quite surprising because the condition numbers grow exponentially in n as 
n — y oo for some large and important classes ofnxn Toeplitz matrices [BG05] , which is opposit to 
the behavior of Gaussian random Toeplitz n x n matrices as we proved and consistently observed in 
our tests. Clearly, our study of Toeplitz matrices can be equally applied to Hankel matrices. 

We organize our paper as follows. We recall some definitions and basic results on general matrix 
computations in the next section and on Toeplitz, Hankel and circulant matrices in Section [3] We 
define Gaussian random matrices and study their ranks and extremal singular values in Section 
2J In Sections [5] and [7] we extend this study to Gaussian random Toeplitz and circulant matrices, 
respectively. In Section [5] we cover numerical tests, which constitute the contribution of the second 
author. In Section |H] we recall some applications of random circulant and Toeplitz matrices, which 
provide some implicit empirical support for our estimates for their condition numbers. We end with 
conclusions in Section [TOl 



2 Some definitions and basic results 

Except for Theorem 13.21 and its application in the proof of Theorem 17.11 we work in the field M of 
real numbers. Next we recall some customary definitions of matrix computations [GL96 , |S98]. A T 
is the transpose of a matrix A. \\A\\h is its /i-norm for h = 1,2, oo. We write ||^4|| to denote the 
2-norm ||^4||2- We have 

! <PH<^PHi, ||A|U = Halloo, UfKWAMAlU (2.1) 



>m 

for an m x n matrix A, 

\\AB\\h < \ \A\\h\\B\\h for h = 1, 2, oo and any matrix product AB. (2-2) 
Define an SVD or full SVD of an m x n matrix A of a rank p as follows, 

A = Sa^aTJ. (2.3) 

Here S A S\ = S^S A = I m , T A Tj = T\T A = 7„, E A = diag(SA, O m -p, n -p), £U = diag(cr,(A))^ =1 , 
tjj = (Jj(A) = tjj{A T ) is the jth largest singular value of a matrix A for j = 1, . . . , p, and we write 
<7j = for j > p. These values have the minimax property 

ff j = max . min | \Ax\ |, j = 1, . . . , p, (2.4) 

dim(s)— j x^S, ||x|| = l 

where § denotes linear spaces [GL961 Theorem 8.6.1]. 

Fact 2.1. If Aq is a submatrix of a matrix A, then o~j(A) > aj(Ao) for all j. 

Proof. [GL961 Corollary 8.6.3] implies the claimed bound where Ao is any block of columns of the 
matrix A. Transposition of a matrix and permutations of its rows and columns do not change 
singular values, and thus we can extend the bounds to all submatrices Aq. □ 

A + = T A diag(E^ 4 1 , O n - P}m - p )S^ is the Moore-Penrose pseudo-inverse of the matrix A of (|2.3|) . 
and 

\\A+\\ = l/a p (A) (2.5) 

for a matrix A of a rank p. 

k(A) = p i^j = \\A\\ ||A + || is the condition number of an m X n matrix A of a rank p. Such 
matrix is ill conditioned if <Ji(A) 3> cr p (A) and is well conditioned otherwise. See |D83j . [GL96, 
Sections 2.3.2, 2.3.3, 3.5.4, 12.5], [H02l Chapter 15], |KL94j . and [S9S Section 5.3] on the estimation 
of the norms and condition numbers of nonsingular matrices. 
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3 Toeplitz, Hankel and /-circulant matrices 



m =i) is defined by its 



l 9)g 



(resp. (^)"L+") 



of dimension 



A Toeplitz m x n matrix T m n = (ti_j). j =1 (resp. Hankel matrices H = (h 

first row and first (resp. last) column, that is by the vector (th)™Zi- 
m + n—1. We write T n = T n . n = (ti-j)™j l =1 (see equation p.ll) below). 

is the ith coordinate vector of dimension n for i — 1, ...,n. The reflection matrix J = 
Jn{^n | ■ ■ • | ei) is the Hankel n x n matrix defined by its first column e„ and its last column ei. 
We have J = J T = J" 1 . 

A lower triangular Toeplitz n x n matrix Z(t) — (tj__j)" J=1 

\n-l w„ /tYaVT _ (7(i-\\T % 



by its first column t = (*fc)£=o- We write ^(t) 7 = (^(*)) 
We have Zv = (w l )" = T 1 and Z(v) 



matrix (see (|3.1|) 
«0 = 0, 



T 



tl 
\tn-l 



t-i 



t-i 
to j 



1 



Zo(v) 



(where = for k < 0) is defined 
= Zq = Z(b2) is the downshift n x n 



YZ^ViZ*- 1 for v = ( Vi )l 



and 



0/ 



i 



Vo 



A 



(3.1) 



0/ 



Combine the equations ||Z(v) 



|Z(v) 



|v||i with (|2.1[) to obtain 



|Z(v) 



< 



Mi- 



(3.2) 



#Pi = efp ^ 0, 



Theorem 3.1. Write T fc = (ti-j)fjl for k = n,n + l. 

(a) Let the matrix T n be nonsingular and write p = T~ e% and q = T^ 1 
then piT- 1 = Z(p)Z(Jq) T - Z(Zq)Z(ZJp) T . 

In parts (b) and (c) below let the matrix T n+ \ be nonsingular and write v = (v;)"=o = T^Kex, 
v = (vi)^o 1 , v' = (vi)2 =1 , w = (Wi)2 =0 = T-^e n+1 , w = (w^o, and w' = «)? =1 . 

(b) If vq ^ 0. i/ien the matrix T n is nonsingular and voT n 1 = Z(v)Z(Jw') T — Z(w)Z(Jv') T . 

(c) If v n ^ 0, then the matrix Ti.o = (ti—j)^i j—o * s nonsingular and v n T^ = Z(w)Z(Jv') T — 
Z(v)Z(JW) T . 

Proof. See [GS72] on parts (a) and (b); see |GK72j on part (c). 



□ 



Zf = Z + fe[e n for a scalar / 7^ denotes the n x n matrix of f-circular shift (see p.l[l ). 
An f-circulant matrix Zf(y) = X)"=i v iZ % jT 1 is a special Toeplitz n x n matrix defined by its first 
column vector v = (w^)™ =1 and a scalar /. /-circulant matrix is called circulant if / = 1 and skew 
circulant if / = — 1. By replacing / with we arrive at a lower triangular Toeplitz matrix Z(y). 
The following theorem implies that the inverses (wherever they are defined) and pairwise products 
of /-circulant n x n matrices are /-circulant and can be computed in O(nlogrt) flops. 

Theorem 3.2. (See \CPW74[ ) We have Z x (v) = Q- l D(Qv)Q,. More generally, for any f ^ 0, 
we have Z/»(v) = Uj L D(U f v)U f where U f = QD(f), f = (f 1 )^ , D(u) = diag(ui)£o for a 
vector u = (tij)jT , Q — (u^)"j_ is the n x n matrix of the discrete Fourier transform at n points, 
uj n = cxp(^ 



T) being a primitive n-th root of 1, and Q 
Hankel m x n matrices H 



k( ul -ij) n - 1 — If) 



H 



'1,3 



(ftt+jOijW can be defined equivalently as the products H 
or H = J m T ofmxn Toeplitz matrices T and the Hankel reflection matrices J = J m or J n 
that J = J- 1 = J T and obtain the following simple fact. 



= TJ n 
Note 



Fact 3.1. For m = n we have T = HJ, H- 1 = JT 



anc 



d T = JH if H = TJ, whereas 



T = JH, H- 1 = JT~ 



id T = H J if H = JT . Furthermore in both cases n(H) = k(T). 



By using the equations above we can readily extend any Toeplitz matrix inversion algorithm 
to Hankel matrix inversion and vice versa, preserving the flop count and condition numbers. E.g. 
(JT)- 1 = T- 1 J, (TJ)- 1 = JT' 1 , (JH)- 1 = H- X J and (HJ)- 1 = JH' 1 . 



3 



4 Gaussian random matrices and their ranks 



Definition 4.1. F 1 (y) — Probability^ < y} (for a real random variable 7 J is the cumulative dis- 
tribution function (cdf) 0/7 evaluated at y. F g ^_ a -j(y) = ^ f-00 ex P(~ for a Gaussian 
random variable g(fi, a) with a mean /1 and a positive variance a 2 , and so 

fj, — 4a < y < (i + Aa with a probability near 1. (4-1) 

Definition 4.2. A matrix (or a vector) is a Gaussian random matrix (or vector) with a mean /i 
and a positive variance a 2 if it is filled with independent identically distributed Gaussian random 
variables, all having the mean \i and variance a 2 . G™£ n is the set of such Gaussian random m x n 
matrices (which are standard for ji = and a 2 = 1). By restricting this set to Toeplitz or f-circulant 
matrices we obtain the sets T™* n and -Z?^™ of Gaussian random Toeplitz and Gaussian random 
/-circulant matrices, respectively. 

Definition 4.3. Xn,<r,n(y) is the cdf of the norm ||v|| = (Y17=i v 'i) 1 ^ 2 °f a Gaussian random vector 
v = Oi)"=i € G^ 1 - Fory>0 we have xa,i,n{y) = 2 "/^r(n/2) Jo a; "~ 1 exp(-x 2 /2)dx where T(h) = 
J °° x' 1-1 exp(— x)dx, T(n + 1) = n\ for nonnegative integers n. 

The total degree of a multivariate monomial is the sum of its degrees in all its variables. The 
total degree of a polynomial is the maximal total degree of its monomials. 

Lemma 4.1. \DL1S{ , \S80§ , \Z19\ . For a set A of a cardinality |A| in any fixed ring let a polynomial 
in to variables have a total degree d and let it not vanish identically on this set. Then the polynomial 
vanishes in at most d|A| m_1 points. 

We assume that Gaussian random variables range over infinite sets A, usually over the real line 
or its interval. Then the lemma implies that a nonzero polynomial vanishes with probability 0. 
Consequently a square Gaussian random general, Toeplitz or circulant matrix is nonsingular with 
probability 1 because its determinant is a polynomials in the entries. Likewise rectangular Gaus- 
sian random general, Toeplitz and circulant matrices have full rank with probability 1. Hereafter, 
wherever this causes no confusion, we assume by default that Gaussian random general, Toeplitz 
and circulant matrices have full rank. 



5 Extremal singular values of Gaussian random matrices 

Besides having full rank with probability 1, Gaussian random matrices in Definition 14.21 are likely 
to be well conditioned [D88j , [E88] . [ES05] . |CD05j . [BIT] , and even the sum M + A for M £ R m xn 
and A £ G™„ n is likely to be well conditioned unless the ratio cr/||M|| is small or large [SST06 . 

The following theorem states an upper bound proportional to y on the cdf (2/)> that is 

on the probability that the smallest positive singular value 1/||^4 + || = o~i{A) of a Gaussian random 
matrix A is less than a nonnegative scalar y (cf. (|2.5[1 ) and consequently on the probability that the 
norm ||^4 + || exceeds a positive scalar x. The stated bound still holds if we replace the matrix A by 
A — B for any fixed matrix B, and for B — O m , n the bounds can be strengthened by a factor y\ m ~ n \ 
[ES05] . [CDlfe) . 

Theorem 5.1. Suppose A £ G™* n , B £ R mxn , I = min{m,77}, x > 0, and y > 0. Then 
F ai (A-B)(y) < 2.35 Vly/cr, that is Probability{\\{A - B)+\ \ > 2.35x v / I/cr} < 1/x. 

Proof. For m = n this is [SST061 Theorem 3.3]. Apply Fact [O to extend it to any pair {m, 77}. □ 

The following two theorems supply lower bounds i 7 ] |a||( 2 ) an d F K ^(y) on the probabilities that 
1 1 < z and k{A) < y for two scalars y and z, respectively, and a Gaussian random matrix A. We 
do not use the second theorem, but state it for the sake of completeness and only for square n x n 
matrices A. The theorems imply that the functions 1 — ^1 1 A| | C-^) an d 1 — F K ^(y) decay as z — > 00 and 
y — > 00, respectively, and that the two decays are exponential in —z 2 and proportional to \/\ogy/y, 
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respectively. For small values yo~ and a fixed n the lower bound of Theorem 15.31 becomes negative, 
in which case the theorem becomes trivial. Unlike Theorem 15.11 in both theorems we assume that 
li = 0. 

Theorem 5.2. WSOll Theorem II.7]. Suppose A £ , h = max{m,n} and z > 2ay/h. Then 

F\\A\\( Z ) > 1 — exp(— (z — 2o~y/~h) 2 / (2o~ 2 )) , and so the norm \\A\\ is likely to have order o\fh. 

Theorem 5.3. \SST0b\ Theorem 3.1]. Suppose < a < 1, y > 1, A £ Qq * n . Then the matrix A 
has full rank with probability 1 and F K (^A){y) > 1 — (14.1 + 4.7-y/ (2 In y) /n)n/(yo~). 

Proof. See [SST061 the proof of Lemma 3.2]. □ 

6 Extremal singular values of Gaussian random Toeplitz ma- 
trices 

A matrix T n = (£^_ J - )" J=1 is the sum of two triangular Toeplitz matrices 

T n = Z(t) + Z(t_ f, t = (U)^ \ t = (t'_^- Q \ t' = 0. (6.1) 
If T n £ 7^™^"! then T n has 2n — 1 pairwise independent entries in Gn,<r- Thus (|3.2|l implies that 



iit„h < nz(t)n + \\z(t-y ii < Htiu + Ht-id = n&)r=T-jii < V2^t ii(^)?=i- T 

Recall Definition 14.21 and obtain 



F\\T„\\(y) > x M ,<r,2n-i(y/V2n-l). (6.2) 
Next we estimate the norm 1 1 1 1 1 for T n £ T?* n . 

Lemma 6.1. [SST06, Lemma A. 2]. For a nonnegative scalar y, a unit vector t £ R nxl 7 and a 
vector b £ , we have F\ t T b \(y) < 

Remark 6.1. The latter bound is independent of fi and n; it holds for any fi even if all coordinates 
of the vector b are fixed except for a single coordinate in Gfj.,a- 

Theorem 6.1. Given a matrix T n = (jti—j)i j = \ £ , assumed to be nonsingular (cf. Section 

[^]), write pi — ejT~ 1 ei. Then F^n.rp-iiAy) < 2na.fi for two random variables a and (3 such that 

F a (y) < \t^- and Fp(y) < \^ for y > 0. (6.3) 

V 7T a \ IT 0~ 

Proof. Recall from part (a) of Theorem O that piT,- 1 = Z(p)Z(Jq) T - Z(Zq)Z(ZJp) T . Therefore 
IbiT" 1 !! < ||Z(p)|| ||Z(Jq) T || + \\Z(Zq)\\ \\Z(ZJp) T \\ for p = T^e u q = T~ X B n , and pi = p T ei . 
It follows that | \piT~ 1 1 1 < ||Z(p)|| ||Z(Jq)|| + ||Z(Zq)|| \\Z(ZJp)\\ since \\A\\ = \\A T \ | for all matrices 
A. Furthermore HpiT^ 1 !! < ||p||i || Jq||i + ||Zq||i ||ZJp||i due to (|3"^]l . Clearly ||Jv||i = ||v||i and 
||Zv||i < ||v||i for every vector v, and so (cf. (|2.1|l ) 

IbiT" 1 !! <2||p|| 1 ||q|| 1 <2n||p|| ||q||. (6.4) 

By definition the vector p is orthogonal to the vectors T n e2, ■ ■ ■ , T n e n , whereas p T T„ei = 1 (cf. 
SST06]). Consequenty the vectors T„e 2 , . . . ,T n e n uniquely define the vector u = p/||p||, whereas 
|u T T„ei| = l/||p||. The last coordinate t n -\ of the vector T n ei is independent of the vectors 
T n e2, ■ ■ ■ , T n e n and consequently of the vector u. Apply Remark 16.11 to estimate the cdf of the 

random variable a = l/||p|| = |u T T n ei| and obtain that F a {y) < y ^ff for y > 0. 



Likewise the n — 1 column vectors Tei, . . . , T„_i define the vector v = /3q for (3 = f/||q|| = 
|v T r„e n |. The first coordinate t\- n of the vector T n e n is independent of the vectors Te±, . . . , T„_i 
and consequently of the vector v. Apply Remark 16.11 to estimate the cdf of the random variable (3 

and obtain that Fp(y) < for y > 0. Finally combine these bounds on the cdfs F a {y) and 

F p (y) with □ 

By applying parts (b) and (c) of Theorem 13.11 instead of its part (a) , we similarly deduce the 
bounds ||i'oT , r 7+ill < 2a/3 and H^nT"^! < 2a/3 for two pairs of random variables a and /3 that 
satisfy (|6.3p for n + 1 replacing n. We have p\ = d ^ ( I^~ 1 , «o = ,^ C * T " , and n„ = ^f-J 0,1 for 



dctT„ ' " u — dctT„ + i' t/ ™ ~~ detT„ + i 

2o,i = (U-j)i^ojLi- Next we bound the geometric means of the ratios | detx +1 1 f° r ^ = 1, • • • , fc— 1- 
l/|pi| and 1 / 1 f o | are such ratios for k = n — 1 and k = n, respectively, whereas the ratio l/|f n | is 
similar to l/|t>o|, under slightly distinct notation. 

Theorem 6.2. Let Th ^ O denote h x h matrices for h = l,...,fc whose entries have absolute 
values at most t for a fixed scalar or random variable t, e.g. for t — ||T||. Furthermore let T\ = (t). 
Then the geometric mean {UhZi fdlrT l) 17 ^"^ = ll detTk] 1 /^-^ is at most k^ 1+ T^k. 

Proof. The theorem follows from Hadamard's upper bound |detM| < fc fc / 2 £ fc , which holds for any 
k x k matrix M = [niij)^j =1 with max^ J _ 1 \mij\ <t. □ 

The theorem says that the geometric mean of the ratios | det Th+i / det Th | for h = 1, . . . , k — 1 
is not greater than fe°- 5 + e wt where e(fc) — > as k ~ > oo. Furthermore if T n £ T^ 1 *™ we can write 
t = \ \T\\ and apply to bound the cdf of t. 



7 Extremal singular values of Gaussian random circulant ma- 
trices 

Next we estimate the norms of a random Gaussian /-circulant matrix and its inverse. 



Theorem 7.1. Assume y > and a circulant n x n matrix T — Z\{y) for v G • Th 



en 



( a ) F\\ T \\{y) > X^aAJ^y) for Xn,<r,n(y) in Definition\Jlj\ and (b) F in \ T -i\\(y) < 



fi ,a 

2 ny 
7r a 



Proof. For the matrix T = Z\(y) we have both equation (|6.1[) and the bound ||t_||i < ||t||i, and 
so ||T||i < 2||t||i. Now part (a) of the theorem follows similarly to (I6.2j) . To prove part (b) recall 
Theorem O and write B = ilTQ,- 1 = D(u), u = (m*)"^ 1 = We have °~j( T ) = a j( B ) for a11 3 
because and y/nfl^ 1 are unitary matrices. By combining the equations Ui — efilv, the bounds 

||3?(e^r2)|| > 1 for all i, and Lemma I5TT1 deduce that F\s/}( Ui )\(y) < y^ff for i = 1, . . . , n. We have 
Fa n (B)(y) = F mirii \ui\iv) because B = diag(u i )" = T 1 , and clearly |uj| > |5R(uj)|- □ 



Remark 7.1. Our extensive experiments suggest that the estimates of Theorem \7.1\ are overly pes- 
simistic (cf. Table\8J$. 



Combining Theorem 13 . 2 1 with minimax property (|2.4p implies that 

1 



9(f) 



^(^(v)) < a^Zfiv)) < giffaiZtW) 



for all vectors v, scalars / ^ 0, g{f) — max{|/| 2 , 1/|/| 2 }, and j = l,...,n. Thus we can readily 
extend the estimates of Theorem 17.11 to /-circulant matrices for / ^ 0. In particular Gaussian 
random /-circulant matrices tend to be well conditioned unless / sa0orl//w0. 
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8 Numerical Experiments 



Our numerical experiments with random general, Hankel, Toeplitz and circulant matrices have been 
performed in the Graduate Center of the City University of New York on a Dell server with a dual 
core 1.86 GHz Xeon processor and 2G memory running Windows Server 2003 R2. The test Fortran 
code was compiled with the GNU gfortran compiler within the Cygwin environment. Random 
numbers were generated with the random_number intrinsic Fortran function, assuming the uniform 
probability distribution over the range {x : —1 < x < 1} . The tests have been designed by the first 
author and performed by his coauthors. 

We have computed the condition numbers of random general n x n matrices for n = 2 fe , 
k = 5,6,..., with entries sampled in the range [—1,1) as well as complex general, Toeplitz, and 
circulant matrices whose entries had real and imaginary parts sampled at random in the same range 
[—1,1). We performed 100 tests for each class of inputs, each dimension n, and each nullity r. Tables 
I8.2H8.4I display the test results. The last four columns of each table display the average (mean), min- 
imum, maximum, and standard deviation of the computed condition numbers of the input matrices, 
respectively. Namely we computed the values k(A) = \ \A\\ ||^4 _1 || for general, Toeplitz, and circu- 
lant matrices A and the values K\{A) — \\A\\\ for Toeplitz matrices A. We computed and 
displayed in Table [831 the 1-norms of Toeplitz matrices and their inverses rather than their 2-norms 
to facilitate the computations in the case of inputs of large sizes. Relationships (|2.1j) link the 1-norms 
and 2-norms to one another, but the empirical data in Table I87T1 consistently show even closer links, 
in all cases of general, Toeplitz, and circulant n x n matrices A where n = 32, 64, . . . , 1024. 



Table 8.1: The norms of random general, Toeplitz and circulant n x n matrices and of their inverses 



matrix A 


n 


U\\i 


1141a 




A 


l 








A~ 


1 




A 


2 




A- 1 


2 


General 


32 


1.9 x 10 1 


1.8 x 10 1 


1.0 x 10 u 


4.0 x 10 2 


2.1 x 10 2 


1.9 x 10° 


General 


64 


3.7 x 10 1 


3.7 x 10 1 


1.0 x 10" 


1.2 x 10 2 


6.2 x 10 1 


2.0 x 10° 


General 


128 


7.2 x 10 1 


7.4 x 10 1 


9.8 x lO" 1 


3.7 x 10 2 


1.8 x 10 2 


2.1 x 10 u 


General 


256 


1.4 x 10 2 


1.5 x 10 2 


9.5 x lO" 1 


5.4 x 10 2 


2.5 x 10 2 


2.2 x 10° 


General 


512 


2.8 x 10 2 


3.0 x 10 2 


9.3 x lO" 1 


1.0 x 10 a 


4.1 x 10 2 


2.5 x 10 u 


General 


1024 


5.4 x 10 2 


5.9 x 10 2 


9.2 x lO" 1 


1.1 x 10 3 


4.0 x 10 2 


2.7 x 10° 


Toeplitz 


32 


1.8 x 10 1 


1.9 x 10 1 


9.5 x lO" 1 


2.2 x 10 1 


1.3 x 10 1 


1.7 x 10 u 


Toeplitz 


64 


3.4 x 10 1 


3.7 x 10 1 


9.3 x 10- 1 


4.6 x 10 1 


2.4 x 10 1 


2.0 x 10° 


Toeplitz 


128 


6.8 x 10 1 


7.4 x 10 1 


9.1 x 10" 1 


1.0 x 10 2 


4.6 x 10 1 


2.2 x 10 u 


Toeplitz 


256 


1.3 x 10 2 


1.5 x 10 2 


9.0 x lO" 1 


5.7 x 10 2 


2.5 x 10 2 


2.3 x 10° 


Toeplitz 


512 


2.6 x 10 2 


3.0 x 10 2 


8.9 x lO" 1 


6.9 x 10 2 


2.6 x 10 2 


2.6 x 10° 


Toeplitz 


1024 


5.2 x 10 2 


5.9 x 10 2 


8.8 x lO" 1 


3.4 x 10 2 


1.4 x 10 2 


2.4 x 10° 


Circulant 


32 


1.6 x 10 1 


1.8 x 10 1 


8.7 x 10- 1 


9.3 x 10" 


1.0 x 10 1 


9.2 x lO" 1 


Circulant 


64 


3.2 x 10 1 


3.7 x 10 1 


8.7 x 10- 1 


5.8 x 10° 


6.8 x 10 u 


8.6 x lO" 1 


Circulant 


128 


6.4 x 10 1 


7.4 x 10 1 


8.6 x lO" 1 


4.9 x 10° 


5.7 x 10 l) 


8.5 x 10- 1 


Circulant 


256 


1.3 x 10 2 


1.5 x 10 2 


8.7 x 10- 1 


4.7 x 10" 


5.6 x 10 l) 


8.4 x lO" 1 


Circulant 


512 


2.6 x 10 2 


3.0 x 10 2 


8.7 x 10- 1 


4.5 x 10° 


5.4 x 10 u 


8.3 x 10- 1 


Circulant 


1024 


5.1 x 10 2 


5.9 x 10 2 


8.7 x 10- 1 


5.5 x 10" 


6.6 x 10 u 


8.3 x 10- 1 



9 Implicit empirical support of the estimates of Sections [6] 
and m 

The papers |PQa| and |PQZa| describe successful applications of randomized circulant and Toeplitz 
multipliers to some fundamental matrix computations. These applications were bound to fail if the 
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Table 8.2: The condition numbers k(A) of random n x n matrices A 



n 


input 


min 


max 


mean 


std 


32 


real 


2.4 x 10 1 


1.8 x 10 3 


2.4 x 10 2 


3.3 x 10 2 


64 


real 


4.6 x 10 1 


1.1 x 10 4 


5.0 x 10 2 


1.1 x 10 3 


128 


real 


1.0 x 10 2 


2.7 x 10 4 


1.1 x 10 3 


3.0 x 10 3 


256 


real 


2.4 x 10 2 


8.4 x 10 4 


3.7 x 10 3 


9.7 x 10 3 


512 


real 


3.9 x 10 2 


7.4 x 10 b 


1.8 x 10 4 


8.5 x 10 4 


1024 


real 


8.8 x 10 2 


2.3 x 10 b 


8.8 x 10 3 


2.4 x 10 4 


2048 


real 


2.1 x 10 3 


2.0 x 10 5 


1.8 x 10 4 


3.2 x 10 4 



Table 8.3: The condition numbers Ki(A) — ppTjn of random Toeplitz n x n matrices A 



n 


min 


mean 


max 


std 


256 


9.1 x 10 2 


9.2 x 10 3 


1.3 x 10 b 


1.8 x 10 4 


512 


2.3 x 10 3 


3.0 x 10 4 


2.4 x 10 5 


4.9 x 10 4 


1024 


5.6 x 10 3 


7.0 x 10 4 


1.8 x 10 b 


2.0 x 10 b 


2048 


1.7 x 10 4 


1.8 x 10 b 


4.2 x 10 b 


5.4 x 10 b 


4096 


4.3 x 10 4 


2.7 x 10 b 


1.9 x 10 b 


3.4 x 10 b 


8192 


8.8 x 10 4 


1.2 x 10 b ' 


1.3 x 10 7 


2.2 x 10 b 



multipliers were ill conditioned, and so the success gives some implicit empirical support to our 
probabilistic estimates of Sections [5] and [7] and motivates the effort for proving these estimates. 

Namely it is well known that Gaussian elimination with no pivoting fails numerically where the 
input matrix has an ill conditioned leading block, even if the matrix itself is nonsingular and well 
conditioned. In our extensive tests in |PQa| and |PQZa| we consistently fixed this problem by means 
of multiplication by random circulant matrices. This implies that the random circulant matrices 
tend to be nonsingular and well conditioned for otherwise the products would be singular or ill 
conditioned. 

Likewise in other tests in |PQZa| the column sets of the products A T G of an n x m matrix A T 
having a numerical rank p by random Toeplitz m x p multipliers consisently approximated some 
bases for the singular spaces associated with the p largest singular vaues of the matrix A, and this 
was readily extended to computing a rank-p approximation of the matrix A, which is a fundamental 
task of matrix computations [HMTllj . Then again one can immediately observe that these tests 
would have failed numerically if the multipliers and consequently the products were ill conditioned. 

10 Conclusions 

Estimating the condition numbers of random structured matrices is a well known challenge (cf. 
SST06]). We deduce such estimates for Gaussian random Toeplitz and circulant matrices. The 
former estimates can be surprising because the condition numbers grow exponentially in n as n — >• oo 
for some large and important classes of n x n Toeplitz matrices [BG05) . whereas we prove the 
opposit for Gaussian random Toeplitz matrices. Our formal estimates are in good accordance with 
our numerical tests, except that circulant matrices tended to be even better conditioned in the 
tests than according to our formal study. The study of the condition number of Hankel matrices 
is immediately reduced to the study for Toeplitz matrices and vice versa. Can our progress be 
extended to other important classes of structured matrices? 

Acknowledgements: Our research has been supported by NSF Grant CCF-1 116736 and PSC 



Table 8.4: The condition numbers k(A) of random circulant n x n matrices A 



n 


min 


mean 


max 


std 


256 


9.6 x 10° 


1.1 x 10 2 


3.5 x 10 3 


4.0 x 10 2 


512 


1.4 x 10 1 


8.5 x 10 1 


1.1 x 10 3 


1.3 x 10 2 


1024 


1.9 x 10 1 


1.0 x 10 2 


5.9 x 10 2 


8.6 x 10 1 


2048 


4.2 x 10 1 


1.4 x 10 2 


5.7 x 10 2 


1.0 x 10 2 


4096 


6.0 x 10 1 


2.6 x 10 2 


3.5 x 10 3 


4.2 x 10 2 


8192 


9.5 x 10 1 


3.0 x 10 2 


1.5 x 10 3 


2.5 x 10 2 


16384 


1.2 x 10 2 


4.2 x 10 2 


3.6 x 10 3 


4.5 x 10 2 


32768 


2.3 x 10 2 


7.5 x 10 2 


5.6 x 10 3 


7.1 x 10 2 


65536 


2.4 x 10 2 


1.0 x 10 3 


1.2 x 10 4 


1.3 x 10 3 


131072 


3.9 x 10 2 


1.4 x 10 3 


5.5 x 10 3 


9.0 x 10 2 


262144 


6.3 x 10 2 


3.7 x 10 3 


1.1 x 10 b 


1.1 x 10 4 


524288 


8.0 x 10 2 


3.2 x 10 3 


3.1 x 10 4 


3.7 x 10 3 


1048576 


1.2 x 10 3 


4.8 x 10 3 


3.1 x 10 4 


5.1 x 10 3 
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